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Abstract 

Three examples of response surface modeling with CFD are presented for combined cycle propulsion 
components. The examples include a mixed-compression-inlet during hypersonic flight, a hydrogen-fueled 
scramjet combustor during hypersonic flight, and a ducted-rocket nozzle during all-rocket flight. Three different 
experimental strategies were examined, including full factorial, fractionated central-composite, and D-optimal 
with embedded Plackett-Burman designs. The response variables have been confined to integral data extracted 
from multidimensional CFD results. Careful attention to uncertainty assessment and modeling bias has been 
addressed. The importance of automating experimental setup and effectively communicating statistical results 
are emphasized. 

1 Motivation 

Combined-cycle propulsion (CCP) vehicles rely upon the promise of airbreathing-propulsion to improve the impulse 
efficiencies normally associated with conventional chemical-propulsion technologies. Both turbine-based and rocket-based 
combined cycles have received a great deal of attention within the past decade. Turbine-based combined cycle engines have 
the advantage of a very mature low-speed system technology. On the other hand, rocket-based combined-cycle engines tend 
to have a single flowpath used throughout the entire ascent trajectory for a single-stage-to-orbit (SSTO) design. An 
axi symmetric, four-mode RBCC propulsion technology program, known as GTX, has been studied at NASA for the last 
five years. Figure 1(a) shows a reference vehicle concept and FigureJ(b) shows the flowpath in cross-section. 



Design simplicity is a key attribute of the GTX propulsion system, which consists of an axisymmetric engine design. 
Engine operation is defined by four separate modes during an ascent trajectory 1 from earth to orbit. First, the engine 
functions with the rocket ignited in a unique Ejector-Ramjet cycle 2 (mode 1), known as the independent ramjet stream (IRS) 
cycle. Then the rocket engine is switched off and subsonic combustion is present in the ramjet (mode 2). As the vehicle 
continues to accelerate, supersonic combustion occurs in the scramjet (mode 3). The rocket is eventually re-ignited (mode 
4) for the final ascent into orbit in an all-rocket configuration. The single piece of moving hardware, throughout this ascent 
profile, is the translating centerbody of the inlet. Further details on this CCP design and reference vehicle concept are given 
by Trefny 3 . 
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Four different propulsion cycles define a complex trade space. A workable system must optimize overall objectives, as 
a function of separate component performances. This design effort continually refines the GTX concept, based on ever- 
increasing fidelity in the performance analysis. This leads to an expanding level of definition in the GTX propulsion system 
design. The technology maturation phase of the GTX program is directed towards the assessment of sub-systems that cover 
a particular component and/or propulsion cycle. This approach assumes that when pieced together, a comprehensive 
systems-level trade study can be executed. An engineering approach of parametric experimentation and analysis has been 
employed to define sub-system performance over a broad range of operating conditions 4 . A combination of ground test 
experiments, computational fluid dynamic (CFD) simulations, and eventual flight test experiments have been employed to 
achieve this end. Statistical design of experiments (DOE) offers a particularly efficient way to conduct these numerous 
parametric studies. 

2 CFD and Statistical Experiment Design 

2. 1 Experiment Design 

Experiment design is a strategic approach to answering two basic questions about parametric studies: ( 1 ) what inputs 
are specified and (2) what outputs are collected. Although the basic ideas date back to the seminal work of R. A. Fisher, 
some eighty years ago, experiment design has only recently become established within the aerospace engineering 
community . Aerospace engineering is not alone as a latecomer to statistical experiment design, as pointed out in the 
excellent review article of Myers, Khuri and Carter 6 . Several factors have contributed to the relatively limited exploitation 
of experiment designs, including: a historical association with a few specific industries, communication barriers between 
academic statistics and applied engineering, and a historical lag between theoretical developments and software 
capabilities. The last two decades have seen dramatic progress on all three fronts, driven in part by the widespread 
availability of powerful computing hardware and user-friendly statistical software. Myers, Khuri, and Carter also alluded to 
the influence of market forces and “...the recent, almost passionate, push for quality and experimental design in 
industry....’’ The dozen years since those words were written have given rise to the “better, faster, cheaper era” in aerospace 
that has resulted in downward pressure on research budgets across the industry \ The need to maximize “information 
gained per data point’’ has lead to an increased interest in DOE for a variety of aerospace applications. 

A class of designs, known as response surface methods (RSM), have proven useful for revealing the complex and 
interacting effects that often appear in engineering systems. System knowledge can be captured in a polynomial regression 
model. This model can then serve as a surrogate for the complex system response, if properly constructed. These techniques 
are typically thought of as strategies for a “physical” test program (i.e. wind tunnel tests), as opposed to a “simulated” test 
program. The last decade, however, has seen the extension of RSM to computer experiments with impressive results. 

2.2 RSM and Computer Experiments 

The concept of a polynomial RSM based upon “computer experiments” seems to have arisen at least two decades ago 9 
for the calculation of pre-mixed flames and reaction rate coefficients. The literature contains several examples of RSM 
applied successfully to computer experiments. References 10 ' 11,12,13,14,15 demonstrate the variety of aerospace engineering 
analyses that have benefited from this approach. The present work contains three examples that rely upon CFD solutions of 
the Navier-Stokes equations for the response data. However, the deterministic nature of these CFD experiments demanded a 
new approach to error analysis that combined the statistical regression error and acknowledged biases/errors in the CFD 
results. 

Most computer experiments differ from physical experiments in one very significant manner: the computer experiment 
is a deterministic problem, where as the physical experiment is a stochastic problem with measurement uncertainty. This 
attribute calls into question the appropriateness of standard regression-error-analysis for computer experiments, in the 
absence of a normally distributed error 16 . However, traditional lack-of-fit statistics can be used to quantify the agreement 
between the polynomial model itself, and the underlying response data used to build the model. Our approach has been to 
quantify the acknowledged modeling errors (i.e. grid dependence, iterative convergence, turbulence modeling dependence, 
artificial diffusion coefficients, etc.) and combine these with the statistical error (95% confidence interval on future 
prediction of the response). Experience has shown that the construction of a regression model, with prediction errors 
comparable to the modeling errors/biases, can be accomplished. This approach has focused the conclusions upon the trends 
that are clearly significant, when compared to both regression and modeling errors. Confirmation testing of a RSM's 
predictive capability can quickly assess the validity of the general error statement. Section 5 describes this process in some 
detail, although this approach to error analysis has been applied to all three studies discussed below. 
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Statistical science has begun to develop an alternative approach to experimental design for deterministic problems. 
Design and Analysis of Computer Experiments (DACE) is based upon kriging methods from the spatial statistics 
literature 16,17 * 18 * 19 . DACE strategies strive for “space filling” designs, as opposed to classical strategies based upon 
symmetric, rotatable, and optimal form designs. The most striking difference with traditional designs involves the lack of 
design point replication. This gets at the heart of deterministic versus non-deterministic experiment designs: computer 
simulations should be reproducible to machine accuracy. DACE response models differ from traditional (polynomial-based) 
response surface models because they possess the ability to interpolate the data. This feature can be advantageous when the 
deterministic nature of computer experiments is considered. It is often said that DACE methods can “honor the data.” 
However, Papila and Haftka 1 ^ point out that engineering analysis can benefit from the inherent smoothing of polynomial- 
based response models when other numerical noise sources are present. Sasena 20 includes a good general discussion of 
DACE methods, contrasting this approach to several other well-known techniques for global optimization. Giunta and 
Watson 21 make direct comparisons between the polynomial RSM and DACE for several mathematical test functions. 
Simpson et al~~ have compared polynomial RSM and DACE for the multidisciplinary design optimization of an aerospike 
nozzle configuration, using both CFD and structural analysis. These comparisons highlight the complimentary nature of 
these design strategies. In general, the polynomial-based methods will work well when the underlying assumptions of 
smooth variation are present. Predictive capabilities have been similar between the two, for this class of problems. DACE 
methods appear to have a distinct advantage for modeling a design space where extreme non-linearity is expected. 

The objective of this work was to demonstrate how RSM and CFD have been incorporated into a technology 
maturation process for the GTX propulsion system. Three different sub-system models have been discussed: the inlet total 
pressure recovery during hypersonic flight, the fuel mixing efficiency during scramjet mode, and the expansion system 
during ail-rocket mode. Different experimental objectives have led to three different experiment designs. The resulting 
regression models have been incorporated into the overall system-level analysis, albeit in different ways. The inlet recovery 
model was used for a performance map of the hypersonic flight regime, where extensive ground testing was too costly. 
Likewise, the model of fuel mixing was used to screen injector designs for the hypersonic flight regime, where test data was 
difficult to obtain. Finally, the model of specific impulse efficiency was used to develop a performance map and an 
understanding of the governing fluid physics. The paper has been organized into three separate sections. A few common 
themes that relate uniquely to DOE and CFD will be discussed at the conclusion of the paper. 

3 Hypersonic Inlet Analysis: Predicting System Performance 

3. 1 Background 

The GTX inlet (station #1 to #2 of Figure 1(b)) was based upon a mixed-compression design. A small amount of initial 
compression from the vehicle forebody was accounted for as well 2 '. Initial inlet performance of a 12 degree conical design 
has been tested and widely reported 24 * 25 * 26 * 2 . After design modification of the inlet flowpath, a new performance map was 
required for the inlet. Thus a second series of CFD analysis and supersonic wind tunnel testing was initiated. The difficulty 
of hypersonic testing has lead to our reliance upon CFD during this portion of the ascent trajectory. 

3 . 2 Experiment Design 

The emphasis upon highly precise performance prediction has lead to 
a relatively dense full-factorial experiment design in two parameters: 
flight Mach number (M) and inlet contraction ratio (CR). A graphical 
representation of the experimental matrix is shown in Figure 2. The 
primary response variable of this study was a regression model of inlet 
total pressure recovery. The experimental design space covered the 
expected range of practical inlet operations during scramjet mode. The 
freestream Mach number ranged from a six to twelve, while the inlet 
contraction ratio ranged from eight to sixteen. We have assumed that an 
adequate regression model could be assembled from linear, quadratic and 
bi-linear (simple interaction) terms; all higher order terms and interactions 
are assumed to be zero. 

3.3 Numerical Modeling 

The flow solver GASP v3.2 28 from AeroSoft, Inc. was used to conduct the 
non-equilibrium, finite rate analysis. The numerical modeling was similar 
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Figure 2 Experimental design schematic 
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to an earlier study 24 . From the earlier work, the grid resolution requirements for this work were well understood’. The 
present calculations were run on one of three (325 axial by 99 radial) meshes, according to the required contraction ratio. 
The numerical models were the same as documented in the earlier work, with the exception of two boundary conditions. 
The freestream boundary that lies parallel to the incoming flow was either modeled as a slip wall or as a shared block 
interface, depending upon whether the inlet flow spilled. Thus, for a spilled flow situation, resolution of the external flow 
over the cowl was required. The single configuration that spilled was the minimum speed, minimum contraction ratio case 
(i.e., Mach=6, CR=8). The other boundary condition modification was the inclusion of a section of isothermal wall in the 
portion of the internal flow regions. The entire cowl (leading edge to throat) and a portion of the centerbody (cowl 
shockwave to throat) were cooled. The specified wall temperature was 900R at M=6, 1200R at M=9, and 1500R at M=12. 
The output of the CFD calculations was stream thrust averaged, and the stagnation conditions were evaluated with the 
Chemical Equilibrium Analysis code 29 . As noted above, the primary response variable was inlet recovery. A secondary 
response variable, heat transfer per-unit of capture area, was also modeled for this configuration. The data matrix is 
included below in Table 1 . 



Casel 

Case2 

Case3 

Case4 

Case5 

Case6 

Case7 

Case8 

Case9 

Mach # 

6 

6 

6 

9 

9 

9 

12 

12 

12 

C.R. 

8 

12 

16 

8 

12 

16 

8 

12 

16 

Recovery 

69.20% 

69.59% 

68.73% 

29.95% 

26.23% 

24.99% 

12.83% 

9.99% 

9.72% 

Heat 

Trans. 

(btu/ft 2 /s) 

188.76 

230.44 

190.74 

439.65 

559.7 

552.03 

816.64 

1033.0 

1113.9 


Table 1 Inlet design response data 


3.4 Results 


The flow visualizations are summarized below (see Figure 3) in a series of images depicting the shock structures. The 
shock-on-lip condition occurred at the (Mach=6,Contraction Ratio=12) case. The (M=6,CR=8) spills slightly, while the 
remainder of the cases are over-sped to some degree. Detailed discussions about the operation of this inlet are given 
elsewhere; the following discussion will concentrate on the statistical regression results. 

Regression analysis was conducted with Design Expert 30 , and the two models are given separately, below. First the 

— (%) = 235.37 - 35.624 *M - 0.3558 * CR 
recovery model is given by the following formula: p 0i 


+ 1.439* 

with E ~ ±3% @ 95% confidence/ This prediction error estimate is based upon the standard error of regression 


Syjc-‘ 


(dJE) 


, and the Student’s t-distribution 31 . The response surface can be visualized in a 3D carpet plot. The 


graphic results of this response clearly indicate the important role played by Mach number. The relatively small influence of 
contraction ratio can be seen in the data as well. 


* The grid convergence index was (0.928%) for a grid-doubling analysis of the total pressure recovery. For more on the 
GCI, see section 5.3. 

+ This model was the simplest of several very similar regression models that can be fit through the same data. This approach 
has an assumed even distribution of errors, which may raise some concern. An analysis that square-root-transformed the 
recovery data, generates an error estimate that scales with the recovery data itself. 


NASA/TM— 2002-21 1379 


4 



(a) Mach 6 Results 



(b) Mach 9 Results 



(c) Mach 12 Results 

Figure 3 Hypersonic Inlet flow: numerical schlieren 


The heat transfer data was best resolved with a 
square-root transformation. The model form is given by 
the following formula: 

= -7.074 1 4 + 1 .67 1 87 * A/ + 1 .42005 * CR 
-0.083250*0? 2 + 0.098466 *M*CR 


with E ~±0.31 @ 95% confidence. This error statement 
is based upon the standard error of regression, in 


transformed units: 



dominant effect of Mach number was quite visible in both 
the raw data and response model. The interaction of 
contraction ratio and Mach number was, perhaps, an 
unintended complication of the experimental setup. The 
actual amount of cooled wall area differed at each CR 
value. This amounts to a “bookkeeping” issue of where the 
inlet throat occurs, and clouds the proper interpretation of 
contraction ratio effect. One would expect that at a given 
flight condition, a higher contraction ratio would yield a 
higher pressure and temperature environment and thus 
more heat transfer. However, because the amount of 
cooled wall modeled in this study varies in a non-linear 
fashion, this effect was not clearly illuminated with this 
DOE. This issue highlighted the point that a DOE, 
configured with one response in mind, may not adequately 
address a secondary response variable. Below Figure 4 
shows the response surface results for both recovery and 
heat transfer. 

In summary, results indicated that the effect of Mach 
number upon the two response functions (recovery and 
heat transfer) was significant. The effect of contraction 
ratio was small but statistically significant for both the 
recovery and the heat transfer models. Since the statistical 
error was significantly larger than the errors associated 
with either mesh dependence or iterative convergence 
(both under 1%), a general prediction error was set equal 
to the statistical prediction errors, for both models. The 
two models can be used as accurate surrogates for further 
CFD simulation within the experimental design space. 
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4 Scramjet Combustor Analysis: Screening Important Effects 

4.1 Background 

The GTX flowpath presents a design challenge for the air-breathing combustor (see Figure 6(a)). Fuel injection ports 
will be used for both the low speed IRS cycle and the scramjet cycle. Parametric analysis is underway to help define the 
performance implications of different injector design patterns. CFD offers an efficient method, when coupled with ongoing 
experimental efforts, to estimate combustor performance. It is widely recognized that the significant advantage of CFD 
analysis, for scramjet flows, lies in the ability to quickly examine a broad variety of geometric parameters at full scale and 
flight enthalpy. Extensive ground testing of these systems is prohibitively expensive. Of course, CFD has several inherent 
limitations when modeling these flowfields, and results must not be misinterpreted as the definitive answer. CFD has been 
widely used for scramjet design studies^ - to gain insight into this complex trade space. 

The goal of this study was to assess which fuel injector design parameters have the most impact upon the hydrogen fuel 
mixing. Initial CFD analysis of the GTX scramjet combustor has been executed for a variety of conditions'" '. However, after 
reviewing these separate cases, no clear optimum configuration was apparent. Several fuel-injector design parameters 
appeared to have significant, interacting effects upon performance. A second set of CFD calculations, organized around a 
DOE matrix, has been conducted. Unlike the earlier analysis, the present calculations were executed as mixing studies on 
relatively coarse three-dimensional meshes. Mixing analysis implies that no reaction between the hydrogen (H2) and 
oxygen (02) was modeled. This approach was based upon the assumption that the scram combustion process was mixing- 
limited. This approach has been employed to study high-speed fuel injection schemes before with good success 34,35 . 
Another difference from earlier analysis was the reliance upon coarse mesh results for building a response surface model. 
Two levels of mesh refinement have been examined for these problems: a coarse level (280k cells), which is every-other- 
point of a related fine level mesh (2.240M cells). Previous mesh dependence analysis indicated that the coarse results 
underpredict the fine mesh results for combustion efficiency by approximately five-percent. Reliance upon coarse mesh 
results enabled an extensive design space for experimentation, yet restricted the interpretation of the RSM results. 
Conclusions have been limited to (1) screening significant design parameters, and (2) highlighting regions of design space 
for further refinement. These compromises enabled a relatively large study (5 parameters at 3 levels) to be conducted in the 
time allotted, while still modeling the three-dimensional mixing process. 

The concept behind the current scramjet combustor design includes two different fuel injection locations. The first set 
of injectors were placed at station #2 and constituted the “streamwise” injection ports, located in the backstep region of 
both the cowl and centerbody. They were spaced around this annular region at a 1.5-degree pitch. These streamwise 
injectors fueled the flame holding region of the combustor and perhaps, supplied a substantial portion of the required fuel. 
The second set of fuel injectors, wall mounted “transverse” injection ports, were placed in the constant area portion of the 
scram combustor flowpath (see Figure 6(a)). The transverse injectors were located along both the cowl and centerbody at a 
given station, spaced at a three-degree pitch. 
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Note that the CFD domain extended from station #2 to station #3, and simulated a 1.5 degree slice of a full 360-degree 
annular geometry. By contrast, the combustor hardware was approximately 220 degrees, with planar endwalls. We have 
assumed that endwall effects were of secondary importance to this study, and therefore have been neglected. 

4.2 Experiment Design 

The GTX propulsion system has a combustion efficiency target of 92.5% throughout the scramjet mode operation. This 
target efficiency must be achieved at a combustion exit plane that varies throughout the scramjet portion of trajectory. This 
goal was approximated with a mixing efficiency target of 95% at station #3. The definition of mixing efficiency for this 
study is given by the percentage of oxygen that would be present in product water, if the flow was allowed to reach 
chemical equilibrium without further mixing. The species oxygen has been used because all of the conditions within the 
study are stoichiometric or fuel rich. 
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Mach # 
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1 

25% 

15 

fwd 

6.5 

1.4 

94.1 

2 

75% 

75 

fwd 

6.5 

1.4 

63.9 

3 

25% 

75 

fwd 

6.5 

1 

76.0 

4 

75% 

15 

fwd 

6.5 

1 

41.2 

5 

50% 

45 

mid 

6.5 

1.2 

75.7 

6 

75% 

15 

aft 

6.5 

1.4 

41.3 

7 

75% 

75 

aft 

6.5 

1 

44.3 

8 

25% 

15 

aft 

6.5 

1 

61.3 

9 

25% 

75 

aft 

6.5 

1.4 

91.7 

10 

50% 

45 

mid 

9.25 

1 

61.3 

11 

50% 

45 

mid 

9.25 

1.4 

74.1 

12 

25% 

45 

mid 

9.25 

1.2 

84.5 

13 

75% 

45 

mid 

9.25 

1.2 

43.2 

14 

50% 

45 

mid 

9.25 

1.2 

67.5 

15 

50% 

75 

mid 

9.25 

1.2 

71.0 

16 

50% 

45 

aft 

9.25 

1.2 

47.3 

17 

50% 

45 

fwd 

9.25 

1.2 

51.6 

18 

50% 

15 

mid 

9.25 

1.2 

54.5 

19 

25% 

15 

fwd 

12 

1 

75.2 

20 

75% 

75 

fwd 

12 

1 

51.5 

21 

25% 

75 

fwd 

12 

1.4 

86.8 

22 

75% 

15 

fwd 

12 

1.4 

55.3 

23 

50% 

45 

mid 

12 

1.2 

62.8 

24 

75% 

15 

aft 

12 

1 

34.7 

25 

75% 

75 

aft 

12 

1.4 

44.1 

26 

25% 

15 

aft 

12 

1.4 

52.6 

27 

25% 

75 

aft 

12 

1 

47.5 

28 

25% 

45 

mid 

9.25 

1.4 

88.8 


Figure 5 Scramjet mixing study: design schematic Table 2 Experimental matrix for fractionated CCD 
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The emphasis here upon rapid analysis of many potential fuel injector design parameters has lead to a relatively sparse, 
fractionated central composite design (CCD) in five variables (see Figure 5). This specific design was implemented in two 
distinct phases, as represented by the two different colored dots in Figure 5. The black dots represented a fractional- 
factorial-plus-centerpoint (screening) design, while the additional gray dots enabled a more complete response surface 
model. The single clear dot represented the confirmation case (number 28), which is discussed in section 4.4. The five 
design variables were: 

XI: Fuel split between step injectors and wall injectors (25%-75%, 50%-50%, 75%-25%); see Figure 7 
X2: Wall fuel injection angle (75deg, 45deg, 15deg); see Figure 6(c) 

X3: Wall fuel injection placement (fwd, mid, aft); see Figure 6(b) 

X4: Flight Mach number (6.5, 9.25, 12) 

X5: Total fuel/air ratio (1.0, 1.2, 1.4) 

The 17 case screening design was completed and the resulting model revealed 5 significant main effects and 2 interactions. 
A decision was made to continue with the full 27-case study to explore the quadratic effects of these five design parameters. 
The following discussion related specifically to the analysis of the full 27-case response model. Note that several different 
response variables can be modeled from the 3D RANS data of this study, including a variety of mixing efficiencies and loss 
measures. I have confined my remarks to a discussion of the mixing efficiency at station #3. The statistical analysis was 
carried out with Design Expert. 
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(a) Scramjet combustor flowpath (b) Three wall-injector locations (c) Three wall injector angles 

Figure 6 Scramjet combustor geometry 



Figure 7 Hydrogen fuel distribution scheme: (a) 25% step fuel, (b) 50% step fuel, (c) 75% step fuel 


4.3 Numerical Modeling 

The numerical modeling employed for this study was considerably different than prior scramjet analysis' '. The primary 
change was the use of a new version of the GASP solver (version 4.0) 36 . GASP is a 3D, finite volume, structured-mesh 
RANS solver that has been used to analyze many high-speed propulsion flows, including scramjet combustors, in steady 
state or time-dependent fashion. The solver was specified as follows: 

• Time integration: steady state analysis with Gauss-Seidel implicit method and inner iteration scheme (Gauss- 
Seidel, again) 

• Inviscid flux function: Van Leer scheme with MUSCL reconstruction (k=1/3) and the minmod limiter function 

• Laminar transport properties: Sutherland’s law was used for modeling viscosity and thermal conductivity, while 
mass diffusion was modeled as a constant-Schmidt-number flow 

• Turbulent flow model: full 3D viscous analysis, Wilcox '98 k-co model 3 , with turbulence modeling limited as 
follows: n turb < 2000/1 laminar ; turbulent Prandtl and Schmidt numbers are both set to value of 0.5 

• Chemistry model: frozen 3-species (N2, 02, H2) and NASA GRC thermal equilibrium database 

• Boundary conditions: injectors modeled as sonic (fixed condition) H2 jets; combustor walls modeled as adiabatic 
no-slip walls; combustor inflow modeled as supersonic inflow (fixed condition) that was calculated a priori; 
symmetry condition applied at both spanwise boundaries; extrapolation outflow boundary. 

The mesh was slightly finer than the earlier study, and the results were run on a coarse sequence of (140 axial, 100 radial, 
20 spanwise) cells, as discussed above. A fine mesh, used to quantify the mesh dependence, was doubled in each direction 
(280x200x40). The mesh dependence observed at the design centerpoint (case 14) was 4%. This result agreed with earlier 
analysis 33 of mesh dependence for these flowfields. The geometry was modeled at the reference vehicle scale, as opposed to 
the (8.5%) model scale used in the earlier study. All results have been converged so that massflux was constant to within 
(±1%). This convergence takes approximately 4500 iterations on the coarse mesh, and another 2500 iterations on the fine 
mesh. The coarse mesh result took approximately 900 cpu hours (SGI R12k 400mhz) on a 16-cpu 02000 machine. This 
translated into approximately 2.3 days of wall-clock time. 

4.4 Results 

Figure 8 shows three examples of the three-dimensional mixing results within the dataset. The images depict the 
flowpath (between stations two and three) with cross-section slices of local fuel/air ratio. The small dark ports on the 
centerbody geometry clearly indicate the location of the wall injection holes. The cowl (not shown) had similarly located 
injection ports. The regions with fuel-rich conditions appeared as black; likewise the white regions depicted the complete 
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absence of any fuel. For effective fuel mixing, one would expect to see a uniformly dark cross section at the combustor exit 
plane. Poorly mixed results tended to concentrate the hydrogen fuel in the near-wall region, which resulted in a band of 
white (unmixed oxygen) along the centerline of the flowpath. The natural interpretation of this stratified result was poor 
fuel-jet penetration into the high-speed core. 




The observed station #3 mixing efficiencies varied from 34%-94%. This broad range of “percentage” results required a 




(a) Case #1: 94% mixing efficiency (b) Case #14: 68% mixing efficiency (c) Case #24: 34% mixing efficiency 
Figure 8 Three-dimensional flowfield contours of local fuel-air ratio 


transformation of the data, prior to modeling. The transformation was of the form: In 


Tin,* 

100 % - 77 ^ 


. This confined the 


mixing efficiency model to the meaningful range of [0,100%]. 

The regression model that resulted from this work contained five significant linear effects, two quadratic effects and four 
bilinear interactions. Several significant trends are described below in a series of three carpet plots. Overall, the regression 
model pointed to a distinct region in design space that deserves further refinement and optimization. 

Figure 9 revealed two of the strongest trends in the data: the influence of Mach number and total fuel/air ratio upon 
mixing performance. In general, mixing performance improved with increasing levels of (<j)). This trend is well known for 
scramjet combustors. Likewise, the performance was observed to 
be inversely proportional to ffeestream Mach number. Figure 9, 
which cut through a region of high mixing efficiency within the 
design hyperspace, suggested that the target of 95% was only 
realized at the minimum Mach number and maximum fuel/air 
ratio. 

Figure 10 reveals another strong trend in the data: the 
influence of the fuel distribution between streamwise and 
transverse injectors. For all Mach numbers, performance was 
maximized when the majority of the H2 was injected thru the 
transverse injectors. This result was intuitive, because a jet-in- 
crossflow has better near-field mixing than a comparable co- 
flowing jet. 

Figure 1 1 revealed another important aspect of the data: the 
minor influence of transverse injection angle and location. 

Figure 1 1 depicted the mixing efficiency as a function of the 
transverse injection angle and location for three different Mach 
number conditions: the white surface represented Mach 6.5 flow, 
the gray surface represented Mach 9.25 flow and the dark 
surface represented Mach 12 flow. Notice that the response 
surfaces were relatively flat for the region defined by the fwd- 
mid injector location, at all injection angles (15-75deg). This 
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implied that injector definition within these bounds could be 
driven by other performance constraints, such as minimization 
of total pressure loss. One might expect the dramatic penalty 
observed for injecting the fuel at a minimum angle (low 
penetration) from the furthest point downstream (aft end of 
constant area combustor). This configuration had little chance 
of fueling the high-speed core flow. Increasing Mach number 
appeared to exacerbate this penalty. The primary conclusion, 
drawn from the trends evident in Figure 1 1, was this: a single 
transverse injector design (location and angle) can be used to 
achieve relatively good mixing performance across the Mach 
number range. 

Thus far, confirmation of the predictive capability of this 
response model has been examined at one location within 
design space (see case 28 of table 2). The observed CFD result 
(88.8% mixing efficiency) was in good agreement with the 
RSM prediction of (86.9%), and well within the ninety-five- 
percent confidence interval of [79. 1%,92. 1%]. This result 
merely suggests that the response model can serve as a 
surrogate for the coarse mesh results. In general, the 95% 
confidence interval for prediction was neither constant nor 
symmetric within design space. This was a result of the 
natural-logarithm transformation. Since the response model was built upon data with a significant mesh dependency, it 
should not be used as a surrogate for future CFD results within this entire design space. Rather, a second RSM analysis 
should be conducted in the region of high mixing efficiency with fine mesh resolution. This second model should yield a 
highly reliable surrogate appropriate for design optimization. 

In summary, a polynomial model of mixing efficiency has been constructed from 27 coarse mesh solutions of a 3D 
scramjet combustor. The study included five parameters important to fuel injector design. All five parameters appeared as 
significant factors in the response model of mixing efficiency. The mixing efficiencies were observed to span the range 
from 34% to 94%. The fidelity of the coarse mesh simulations 
was adequate for screening the design variables and 
identifying regions of high mixing efficiency. A distinct region 
in design space appeared to hold the most promise for 
maximizing the mixing efficiency, across the Mach number 
range. This region was defined as follows: 

• Fuel split between step and wall injectors: 25% 
step and 75% wall, or XI set to minimum value. 

• Wall injection angle: 15 to 75 degrees, or X2 
unconstrained. 

• Wall injection placement: fwd to mid location, 
or X3 constrained to lower-half of design space. 

• Fuel-air ratio: 1 .4, or X5 set to maximum value. 

An analysis of total pressure loss within the flowfield is 
expected to narrow the region of interest for injector location 
and angle (X2 and X3). Follow-on analysis can now focus 
upon a reduced design space (Mach number and fuel-air ratio) 
to develop a performance map of mixing efficiency in the 
region of interest. One would expect the results from the next 
phase to be more tightly clustered around 95% efficiency. This 
should greatly reduce the ninety-five-percent confidence 
interval of prediction, while demanding a higher level of 
numerical fidelity in the CFD analysis. The fine-mesh analysis 
is required for this next design iteration, which can be 
accomplished in five CFD experiments. 


injector angle deg 
60 


100 


n 



injector location^ 


Figure 11 Scramjet mixing response surface: 
mixing efficiency as a function of transverse 
injector angle and location at Mach 6.5 (white), 
Mach 9.25 (gray) and Mach 12 (dark) 
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5 Rocket-Mode Performance: Prediction and Uncertainty Analysis 

5. 1 Background 

Vehicle performance is highly dependent upon the efficiency 
achieved during the fourth and final propulsion mode. This is due 
to the fact that the largest portion of the ascent trajectory is flown 
with the inlet closed and the rocket turned on. However, the 
literature on this all-rocket mode is relatively sparse. Most studies 
of RBCC systems have focused their experimental efforts upon the 
air-breathing sub-system performances, and assumed some value of 
expansion efficiency for the all-rocket mode. This was primarily 
due the costs associated with simulating this high-speed, high 
altitude flow with a ground test. We have chosen to initially 
investigate this fourth mode with CFD. We attempted to reveal the 
most critical design parameters for further investigation and 
optimization. A generic RBCC engine geometry was specified with 
similar attributes to the GTX system (see Figure 12). The emphasis 
here upon screening a large number of potential design parameters 
lead to a 36-case D-optimal experiment design in six variables. The 
specific objectives were to: 

1 . Determine the most critical design parameters that 
influence performance for further analysis. 

2. Develop an initial predictive capability for impulse 
efficiency for incorporation in the GTX design cycle. 

3. Apply flow visualization to examine the predominant fluid 
dynamic effects. 

The axisymmetric, Reynolds-averaged Navier-Stokes 
simulations were executed with the NPARC v3.0 code. A response 
surface was created for specific impulse (I sp ) efficiency, based upon 
data integrated from the 36 CFD simulations. Thirteen statistically 
significant effects were observed to have an influence on RBCC 
nozzle performance. The results of this effort have been widely 
reported 1 ’. The following discussion outlines the uncertainty analysis undertaken. This analysis incorporated several 
sources of modeling bias together with the standard error of regression, to develop an overall uncertainty bound on the 
precision of this effort. Our goal was to report an uncertainty bound for the response model that was representative of these 
acknowledged errors. 

At the outset of this study, the authors sought to avoid modeling effects that appeared in the results at a level associated 
with grid dependence, choice of turbulence model, selection of convergence criterion, and specification of numerical 
viscosity coefficients. These were assumed to be the critical components associated with “noise” in the results. Our goal 
was to quantify the influence of these items, and seek to fit a regression model through the data with a predictive error that 
approximated this “noise” level. This was an informal approach to insuring that our results were robust to the arbitrary 
specification of these numerical modeling parameters. The advantage of this approach was that it permitted the familiar 
error statement and confidence interval to be defined according to classical theory. The disadvantage was the implied 
assumption of a stochastic response, which is not strictly valid. An unavoidable consequence of publishing an approximate 
predictive error of regression was the need to run some confirmation cases within the design space and examine the 
residuals. 

5.2 Turbulence Modeling Bias 

We quantified the difference in performance prediction, between a robust one-equation model and a widely applied 
two-equation model, with an analysis of variance (ANOVA). A six-parameter screening design (nine-run Plackett/Burman- 
plus-centerpoint) was repeated with each turbulence model to allow a direct assessment. Analysis of Variance (ANOVA) 
techniques were applied and we concluded that the two multiple-linear-regression (MLR) models agreed very well. The 



Inlet dosed 


<■ - Scarfed Rocket Sozzte 


(a) Flowpath geometry for mode 4 (all-rocket) 

-Annular cavity 

-Primary rocket engine 

-Mixer/ejector duct 



(b) Axisymmetric RBCC nozzle geometry 


Figure 12 RBCC nozzle geometry: (a) GTX 
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geometry with four geometric parameters 
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single significant difference was in the definition of the intercept value: the two-equation model consistently predicted 
results that were 1% higher than the one-equation model. 

5.3 Grid Dependence Bias 

Grid dependence is an un-avoidable consequence of modeling continuous physics with discrete mathematics. 
Furthermore, there is a variety of ways to report this artifact of CFD calculations. One approach to uniform reporting mesh 
sensitivity of CFD results has been gaining acceptance by the engineering community 38 . This reporting technique is referred 
to as the Grid Convergence Index. The GCI is not a bound on the error; rather it is “...a reasonable error band, in the flavor 
of a statistician’s 2 a range or an experimentalist’s 20:1 odds....” The GCI accounts for the order-of -accuracy of the CFD 
solver. An evaluation of grid dependence for our primary response variable was assessed at the centerpoint of design space. 
The assessment compared a standard resolution of 56,952 points with a fine resolution of 226,281 points. The raw 
difference in the response was 0.346% and the GCI value was 1 .6%. We have assumed that this value was representative of 
the entire design space, although twenty-seven different grids were used in the study. In order to support this assumption, 
we strictly defined the grid generation process, including the multiblock topology, the minimum near-wall spacing, the 
maximum far-field spacing, and the maximum stretching ratio between adjacent cells. This strict definition of grid 
development enabled similar discretization of flows through different geometries. 

5.4 Iterative Convergence Bias 

Incomplete iterative convergence is a practical reality for engineering applications of CFD. At the outset, we addressed 
the issue of the reported values of I sp and their sensitivity to incomplete iterative convergence. Often overlooked, this issue 
proved to be non-trivial when considering DOE and CFD. Ultimately, we decided to monitor the primary integral 
quantities every 100 iterations. The residuals of massflow and thrust converged at least 4 orders of magnitude. Furthermore, 
we demanded that the value of I sp should not change more than 0. 1 5% over the last 10,000 iterations at a Courant number of 
0.5. This criterion required several of the longest domains to be run in excess of 50,000 iterations, depending upon the 
initial conditions prescribed. The extremely long iteration counts were necessary to resolve the widely varying wave speeds 
present in a domain with large regions of low speed flow present alongside hypersonic jet flow. In general one should be 
able to account for the small, but finite time-dependence of a steady-state response variable. Often it is controlled with 
further iteration upon the numerical solution, so that it is negligible when compared to the other sources of modeling bias. 

5.5 Artificial Diffusion Bias 

The convective terms of the Euler equations were handled with a central difference scheme. The common approach of 
Jamison, Schmidt, and Turkel was applied to introduce second- and fourth-order artificial diffusion to control numerical 
instabilities. The magnitude of artificial diffusion included in a calculation can affect the results. Experience and intuition 
often guide the specification of these particular coefficients because, as Hirsch 39 correctly points out, “...little information 
is known on the stability of general non-linear discretized schemes....” Nevertheless, the influence of artificial diffusion 
upon the response variable is real and can be quantified. For our study, the baseline value of the 4 th -order coefficient was 
(1/100). This was sufficient to stabilize most of the 36 test cases used to build the RSM. An increased value of the 4 th -order 

coefficient (1/32) was required to stabilize 
a few cases and achieve a steady-state 
convergence. The influence of this 
alternate coefficient value upon the 
response was (0.3%). This value was 
considerably less than the bias introduced 
through mesh dependence. 

To summarize, the general error 
statement about predictive capability of 
this RSM incorporated bias errors 
introduced from several modeling 
decisions, as well as the standard error of 
regression. For this study, it was clear that 
these considerations consistently pointed 
to a predictive uncertainty in the range of 
1.1%. This combination of modeling and 
statistical error analysis should focus the 
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target audience upon statistically significant (and reproducible) trends from the data. The results from eight additional 
confirmation cases, shown here in Figure 13, bolstered our conclusions in this regard. 


6 Lessons Learned 

6.1 Communication of DOE Results 

Several common themes have emerged from the studies 
discussed above. One issue of primary importance involved 
effective communication of the complex statistical results. A 
concise discussion of significant interactions and second- 
order effects within a design hyperspace can challenge the 
most gifted of technical writers (and audiences alike). 

However, simply creating a regression calculator with a 
spreadsheet program can allow an interested user to begin 
exploring the DOE results. If a few interesting extremes are 
included as regression examples, then often the important 
trends can be communicated in short order. A more 
involved approach includes the creation of a short program 
for use with one of the many graphical mathematical 
software packages 40 . A third option involves the 
construction of a custom graphic-user-interface (GUI) that 
can graphically portray some important aspects of the 
regression model in real time. Figure 14 shows a basic 
JAVA program used to investigate the RSM results of section 5. This GUI-based approach has several distinct advantages: 

• A single version of the regression model can be made available to a wide audience, via the world-wide-web. 

• Explicit parameter limits can be imposed upon the independent variables, preventing extrapolation. 

• Familiar browser tools (buttons, sliders, etc) can be used to quickly assemble a custom tool. 

Any one of these approaches can be used to make RSM results more approachable to an interested audience. 

6.2 Response Models of Integrated CFD Data 

Application of RSM techniques has thus far been confined to response variables based upon integrated, multi- 
dimensional CFD data. This type of data has an inherently smoother nature than either local or peak-value flow data. 
Integral data is less susceptible to the local nonlinear behavior that is common in multidimensional CFD results. The source 
of this non-linearity can vary from the mesh-generation for complex geometries, to the numerical resolution of shock waves 
and contact discontinuities. 

6.3 Polynomial-based Response Models and UncertaintylBias 

Thus far, the acknowledged modeling bias errors have been approximately the same magnitude as the statistical 
regression uncertainty. In some sense, this has occurred by design because knowledge of the modeling bias was exploited 
during the model building process. In general, the numerical modeling precision should be matched to the expected 
statistical regression errors. Consider the effect of running very costly CFD analysis and encountering a modeling 
uncertainty (i.e. grid dependence) that was far less that the statistical prediction error. This would imply that the mesh 
dependence could have been increased appropriately without affecting the engineering value of the response model. 
Likewise, a RSM prediction error that is substantially smaller than the grid dependence would indicate that further mesh 
refinement is required before the RSM acts as a reasonable surrogate for the CFD solutions. This is precisely what occurred 
in the mixing analysis of section 4. 

Progress with the quantification of modeling errors and uncertainty has received a great deal of attention. The AIAA 
Committee on Standards for CFD has recently published a guide intended to help assess the credibility of modeling and 
simulation in CFD 41 . Shih, Gu, and Chu 42 have helped to develop grid quality measures and error estimates for both 
structured and unstructured meshes. Oberkampf et al 4 ' and colleagues at the Sandia National Labs have begun to present a 
very comprehensive framework for discussing “total uncertainty” in the realm of modeling and simulation. These are but 
three examples of research aimed at quantifying uncertainty and bias for computer experiments. 
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6.4 Computer Experiments and Process Control 

A final comment concerning quality control was relevant to the implementation of these designed experiments. A 
modem CFD solver has a very large set of independent parameters that require explicit specification prior to simulation. 
For instance, a typical namelist-input file for one of the GASP v4 mixing calculations of section 4 included 642 different 
variables; the grid generation process was another separate source of variability. W. Edwards Deming stated that quality 
process control was driven by effectively managing system inputs, and not by mass inspection of system outputs 44 . While 
this lesson was specifically directed at industrial manufacturing processes, it is just as relevant to the successful execution 
of a designed experiment. I have found that automating the setup and execution of these repetitive studies was invaluable 
for controlling inadvertent errors. Simple pre-processing of the experiment can save time and cost by averting mis- 
specification of the input vectors. For instance, a list of schematic drawings that highlight the critical geometric variations 
can help to ensure the fidelity of an intended mesh. An automatically generated check-sheet for setup of flow conditions can 
help to prevent the mis-specification of a simulation, before the computer time is spent. Real-time output of incomplete 
CFD results can also help to flag erroneous runs at an early stage. Efforts spent upon error prevention have been less time 
consuming than error detection. 

7 Conclusions 

Three parametric studies on combined-cycle-propulsion components have been executed in a designed-experiment 
framework. The computer experiments were executed with different Reynolds averaged Navier-Stokes solvers. The multi- 
dimensional CFD datasets were post-processed for integral performance measures, which in turn, were used to assemble a 
response surface model. A response model of CCP component performance can serve as surrogate for further CFD 
experiments, when a design optimization is undertaken. The three response surface models described in the present work 
have contributed to the technology maturation phase of the GTX CCP program at NASA Glenn Research Center. The 
hypersonic inlet recovery model has been used to predict the compression system performance throughout the hypersonic 
portion of the trajectory. The Scramjet mixing model has been used to screen fuel injector designs that maximize fuel-air 
mixing and identify promising regions of design space for further analysis and testing. The model of specific impulse 
efficiency, for RBCC all-rocket mode, has been used to predict system performance and identify performance trends that 
deserve further attention. All three have led to increased fidelity in the GTX propulsion system design and analysis. The 
combination of response surface methods and computational fluid dynamics can provide an important design and analysis 
capability to the development of modem propulsion systems. 
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